Commentary: Writing and Evaluating Qualitative Research Reports

1Division of Public Health, Department of Family and Preventive Medicine, University of Utah,

2Cancer Control and Population Sciences, Huntsman Cancer Institute,

3Department of Pediatrics-Nutrition, USDA/ARS Children’s Nutrition Research Center, Baylor College of Medicine,

4College of Nursing, University of Central Florida,

5Department of Psychiatry and Human Behavior, Brown University, and

6School of Nursing, University of Pennsylvania

Objective To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. Methods A question and answer format is used to address considerations for writing and evaluating qualitative research. Results and Conclusions When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field’s ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health.

The Journal of Pediatric Psychology (JPP) has a long history of emphasizing high-quality, methodologically rigorous research in social and behavioral aspects of children’s health (Palermo, 2013, 2014). Traditionally, research published in JPP has focused on quantitative methodologies. Qualitative approaches are of interest to pediatric psychologists given the important role of qualitative research in developing new theories (Kelly & Ganong, 2011), illustrating important clinical themes (Kars, Grypdonck, de Bock, & van Delden, 2015), developing new instruments (Thompson, Bhatt, & Watson, 2013), understanding patients’ and families’ perspectives and needs (Bevans, Gardner, Pajer, Riley, & Forrest, 2013; Lyons, Goodwin, McCreanor, & Griffin, 2015), and documenting new or rarely examined issues (Haukeland, Fjermestad, Mossige, & Vatne, 2015; Valenzuela et al., 2011). Further, these methods are integral to intervention development (Minges et al., 2015; Thompson et al., 2007) and understanding intervention outcomes (de Visser et al., 2015; Hess & Straub, 2011). For example, when designing an intervention, qualitative research can identify patient and family preferences for and perspectives on desirable intervention characteristics and perceived needs (Cassidy et al., 2013; Hess & Straub, 2011; Thompson, 2014), which may lead to a more targeted, effective intervention.

Both qualitative and quantitative approaches are concerned with issues such as generalizability of study findings (e.g., to whom the study findings can be applied) and rigor. However, qualitative and quantitative methods have different approaches to these issues. The purpose of qualitative research is to contribute knowledge or understanding by describing phenomenon within certain groups or populations of interest. As such, the purpose of qualitative research is not to provide generalizable findings. Instead, qualitative research has a discovery focus and often uses an iterative approach. Thus, qualitative work is often foundational to future qualitative, quantitative, or mixed-methods studies.

At the time of this writing, three of six current calls for papers for special issues of JPP specifically note that manuscripts incorporating qualitative approaches would be welcomed. Despite apparent openness to broadening JPP’s emphasis beyond its traditional quantitative approach, few published articles have used qualitative methods. For example, of 232 research articles published in JPP from 2012 to 2014 (excluding commentaries and reviews), only five used qualitative methods (2% of articles).

The goal of the current article is to present considerations for writing and evaluating qualitative research within the context of pediatric psychology to provide a framework for writing and reviewing manuscripts reporting qualitative findings. The current article may be especially useful to reviewers and authors who are less familiar with qualitative methods. The tenets presented here are grounded in the well-established literature on reporting and evaluating qualitative research, including guidelines and checklists (Eakin & Mykhalovskiy, 2003; Elo et al., 2014; Mays & Pope, 2000; Tong, Sainsbury, & Craig, 2007). For example, the Consolidated Criteria for Reporting Qualitative Research checklist describes essential elements for reporting qualitative findings (Tong et al., 2007). Although the considerations presented in the current manuscript have broad applicability to many fields, examples were purposively selected for the field of pediatric psychology.

Our goal is that this article will stimulate publication of more qualitative research in pediatric psychology and allied fields. More specifically, the goal is to encourage high-quality qualitative research by addressing key issues involved in conducting qualitative studies, and the process of conducting, reporting, and evaluating qualitative findings. Readers interested in more in-depth information on designing and implementing qualitative studies, relevant theoretical frameworks and approaches, and analytic approaches are referred to the well-developed literature in this area (Clark, 2003; Corbin & Strauss, 2008; Creswell, 1994; Eakin & Mykhalovskiy, 2003; Elo et al., 2014; Mays & Pope, 2000; Miles, Huberman, & Saldaña, 2013; Ritchie & Lewis, 2003; Saldaña, 2012; Sandelowski, 1995, 2010; Tong et al., 2007; Yin, 2015). Researchers new to qualitative research are also encouraged to obtain specialized training in qualitative methods and/or to collaborate with a qualitative expert in an effort to ensure rigor (i.e., validity).

We begin the article with a definition of qualitative research and an overview of the concept of rigor. While we recognize that qualitative methods comprise multiple and distinct approaches with unique purposes, we present an overview of considerations for writing and evaluating qualitative research that cut across qualitative methods. Specifically, we present basic principles in three broad areas: (1) study design and methods, (2) analytic considerations, and (3) presentation of findings (see Table 1 for a summary of the principles addressed in each area). Each area is addressed using a “question and answer” format. We present a brief explanation of each question, options for how one could address the issue raised, and a suggested recommendation. We recognize, however, that there are no absolute “right” or “wrong” answers and that the most “right” answer for each situation depends on the specific study and its purpose. In fact, our strongest recommendation is that authors of qualitative research manuscripts be explicit about their rationale for design, analytic choices, and strategies so that readers and reviewers can evaluate the rationale and rigor of the study methods.

Summary of Overarching Principles to Address in Qualitative Research Manuscripts

Qualitative methods are used across many areas of health research, including health psychology (Gough & Deatrick, 2015), to study the meaning of people’s lives in their real-world roles, represent their views and perspectives, identify important contextual conditions, discover new or additional insights about existing social and behavioral concepts, and acknowledge the contribution of multiple perspectives (Yin, 2015). Qualitative research is a family of approaches rather than a single approach. There are multiple and distinct qualitative methodologies or stances (e.g., constructivism, post-positivism, critical theory), each with different underlying ontological and epistemological assumptions (Lincoln, Lynham, & Guba, 2011). However, certain features are common to most qualitative approaches and distinguish qualitative research from quantitative research (Creswell, 1994).

Key to all qualitative methodologies is that multiple perspectives about a phenomenon of interest are essential, and that those perspectives are best inductively derived or discovered from people with personal experience regarding that phenomenon. These perspectives or definitions may differ from “conventional wisdom.” Thus, meanings need to be discovered from the population under study to ensure optimal understanding. For instance, in a recent qualitative study about texting while driving, adolescents said that they did not approve of texting while driving. The investigators, however, discovered that the respondents did not consider themselves driving while a vehicle was stopped at a red light. In other words, the respondents did approve of texting while stopped at a red light. In addition, the adolescents said that they highly valued being constantly connected via texting. Thus, what is meant by “driving” and the value of “being connected” need to be considered when approaching the issue of texting while driving with adolescents (McDonald & Sommers, 2015).

Qualitative methods are also distinct from a mixed-method approach (i.e., integration of qualitative and quantitative approaches; Creswell, 2013b). A mixed-methods study may include a first phase of quantitative data collection that provides results that inform a second phase of the study that includes qualitative data collection, or vice versa. A mixed-methods study may also include concurrent quantitative and qualitative data collection. The timing, priority, and stage of integration of the two approaches (quantitative and qualitative) are complex and vary depending on the research question; they also dictate how to attend to differing qualitative and quantitative principles (Creswell et al., 2011). Understanding the basic tenets of qualitative research is preliminary to integrating qualitative research with another approach that has different tenets. A full discussion of the integration of qualitative and quantitative research approaches is beyond the scope of this article. Readers interested in the topic are referred to one of the many excellent resources on the topic (Creswell, 2013b).

Qualitative research questions are typically open-ended and are framed in the spirit of discovery and exploration and to address existing knowledge gaps. The current manuscript provides exemplar pediatric qualitative studies that illustrate key issues that arise when reporting and evaluating qualitative studies. Example research questions that are contained in the studies cited in the current manuscript are presented in Table 2.

Example Qualitative Research Questions From the Pediatric Literature

There are several overarching principles with unique application in qualitative research, including definitions of scientific rigor and the importance of transparency. Quantitative research generally uses the terms reliability and validity to describe the rigor of research, while in qualitative research, rigor refers to the goal of seeking to understand the tacit knowledge of participants’ conception of reality (Polanyi, 1958). For example, Haukeland and colleagues (2015) used qualitative analysis to identify themes describing the emotional experiences of a unique and understudied population—pediatric siblings of children with rare medical conditions such as Turner syndrome and Duchenne muscular dystrophy. Within this context, the authors’ rendering of the diverse and contradictory emotions experienced by siblings of children with these rare conditions represents “rigor” within a qualitative framework.

While debate exists regarding the terminology describing and strategies for strengthening scientific rigor in qualitative studies (Guba, 1981; Morse, 2015a, 2015b; Sandelowski, 1993a; Whittemore, Chase, & Mandle, 2001), little debate exists regarding the importance of explaining strategies used to strengthen rigor. Such strategies should be appropriate for the specific study; therefore, it is wise to clearly describe what is relevant for each study. For example, in terms of strengthening credibility or the plausibility of data analysis and interpretation, prolonged engagement with participants is appropriate when conducting an observational study (e.g., observations of parent–child mealtime interactions; Hughes et al., 2011; Power et al., 2015). For an interview-only study, however, it would be more practical to strengthen credibility through other strategies (e.g., keeping detailed field notes about the interviews included in the analysis).

Dependability is the stability of a data analysis protocol. For instance, stepwise development of a coding system from an “a priori” list of codes based on the underlying conceptual framework or existing literature (e.g., creating initial codes for potential barriers to medication adherence based on prior studies) may be essential for analysis of data from semi-structured interviews using multiple coders. But this may not be the ideal strategy if the purpose is to inductively derive all possible coding categories directly from data in an area where little is known. For some research questions, the strategy may be to strengthen confirmability or to verify a specific phenomenon of interest using different sources of data before generating conclusions. This process, which is commonly referred to in the research literature as triangulation, may also include collecting different types of data (e.g., interview data, observational data), using multiple coders to incorporate different ways of interpreting the data, or using multiple theories (Krefting, 1991; Ritchie & Lewis, 2003). Alternatively, another investigator may use triangulation to provide complementarity data (Krefting, 1991) to garner additional information to deepen understanding. Because the purpose of qualitative research is to discover multiple perspectives about a phenomenon, it is not necessarily appropriate to attain concordance across studies or investigators when independently analyzing data. Some qualitative experts also believe that it is inappropriate to use triangulation to confirm findings, but this debate has not been resolved within the field (Ritchie & Lewis, 2003; Tobin & Begley, 2004). More agreement exists, however, regarding the value of triangulation to complement, deepen, or expand understanding of a particular topic or issue (Ritchie & Lewis, 2003). Finally, instead of basing a study on a sample that allows for generalizing statistical results to other populations, investigators in qualitative research studies are focused on designing a study and conveying the results so that the reader understands the transferability of the results. Strategies for transferability may include explanations of how the sample was selected and descriptive characteristics of study participants, which provides a context for the results and enables readers to decide if other samples share critical attributes. A study is deemed transferable if relevant contextual features are common to both the study sample and the larger population.

Strategies to enhance rigor should be used systematically across each phase of a study. That is, rigor needs to be identified, managed, and documented throughout the research process: during the preparation phase (data collection and sampling), organization phase (analysis and interpretation), and reporting phase (manuscript or final report; Elo et al., 2014). From this perspective, the strategies help strengthen the trustworthiness of the overall study (i.e., to what extent the study findings are worth heeding; Eakin & Mykhalovskiy, 2003; Lincoln & Guba, 1985).

A good example of managing and documenting rigor and trustworthiness can be found in a study of family treatment decisions for children with cancer (Kelly & Ganong, 2011). The researchers describe how they promoted the rigor of the study and strengthening its credibility by triangulating data sources (e.g., obtaining data from children’s custodial parents, stepparents, etc.), debriefing (e.g., holding detailed conversations with colleagues about the data and interpretations of the data), member checking (i.e., presenting preliminary findings to participants to obtain their feedback and interpretation), and reviewing study procedure decisions and analytic procedures with a second party.

Transparency is another key concept in written reports of qualitative research. In other words, enough detail should be provided for the reader to understand what was done and why (Ritchie & Lewis, 2003). Examples of information that should be included are a clear rationale for selecting a particular population or people with certain characteristics, the research question being investigated, and a meaningful explanation of why this research question was selected (i.e., the gap in knowledge or understanding that is being investigated; Ritchie & Lewis, 2003). Clearly describing recruitment, enrollment, data collection, and data analysis or extraction methods are equally important (Dixon-Woods, Shaw, Agarwal, & Smith, 2004). Coherency among methods and transparency about research decisions adds to the robustness of qualitative research (Tobin & Begley, 2004) and provides a context for understanding the findings and their implications.

In contrast to quantitative research, qualitative research is not typically hypothesis driven (Creswell, 1994; Ritchie & Lewis, 2003). A risk associated with using hypotheses in qualitative research is that the findings could be biased by the hypotheses. Alternatively, qualitative research is exploratory and typically guided by a research question or conceptual framework rather than hypotheses (Creswell, 1994; Ritchie & Lewis, 2003). As previously stated, the goal of qualitative research is to increase understanding in areas where little is known by developing deeper insight into complex situations or processes. According to Richards and Morse (2013), “If you know what you are likely to find, …  you should not be working qualitatively” (p. 28). Thus, we do not recommend that a hypothesis be stated in manuscripts presenting qualitative data.

Consistent with the exploratory nature of qualitative research, one particular qualitative method, grounded theory, is used specifically for discovering substantive theory (i.e., working theories of action or processes developed for a specific area of concern; Bryant & Charmaz, 2010; Glaser & Strauss, 1967). This method uses a series of structured steps to break down qualitative data into codes, organize the codes into conceptual categories, and link the categories into a theory that explains the phenomenon under study. For example, Kelly and Ganong (2011) used grounded theory methods to produce a substantive theory about how single and re-partnered parents (e.g., households with a step-parent) made treatment decisions for children with childhood cancer. The theory of decision making developed in this study included “moving to place,” which described the ways in which parents from different family structures (e.g., single and re-partnered parents) were involved in the child’s treatment decision-making. The resulting theory also delineated the causal conditions, context, and intervening factors that contributed to the strategies used for moving to place.

Theories may be used in other types of qualitative research as well, serving as the impetus or organizing framework for the study (Sandelowski, 1993b). For example, Izaguirre and Keefer (2014) used Social Cognitive Theory (Bandura, 1986) to investigate self-efficacy among adolescents with inflammatory bowel disease. The impetus for selecting the theory was to inform the development of a self-efficacy measure for adolescent self-management. In another study on health care transition in youth with Type 1 Diabetes (Pierce, Wysocki, & Aroian, 2016), the investigators adapted a social-ecological model—the Socio-ecological Model of Adolescent and Young Adult Transition Readiness (SMART) model (Schwartz, Tuchman, Hobbie, & Ginsberg, 2011)—to their study population (Pierce & Wysocki, 2015). Pierce et al. (2016) are currently using the adapted SMART model to focus their data collection and structure the preliminary analysis of their data about diabetes health care transition.

Regardless of whether theory is induced from data or selected in advance to guide the study, consistent with the principle of transparency, its role should be clearly identified and justified in the research publication (Bradbury-Jones, Taylor, & Herber, 2014; Kelly, 2010). Methodological congruence is an important guiding principle in this regard (Richards & Morse, 2013). If a theory frames the study at the outset, it should guide and direct all phases. The resulting publication(s) should relate the phenomenon of interest and the research question(s) to the theory and specify how the theory guided data collection and analysis. The publication(s) should also discuss how the theory fits with the finished product. For instance, authors should describe how the theory provided a framework for the presentation of the findings and discuss the findings in context with the relevant theoretical literature.

A study examining parents’ motivations to promote vegetable consumption in their children (Hingle et al., 2012) provides an example of methodological congruence. The investigators adapted the Model of Goal Directed Behavior (Bagozzi & Pieters, 1998) for parenting practices relevant to vegetable consumption (Model of Goal Directed Vegetable Parenting Practices; MGDVPP). Consistent with the adapted theoretical model and in keeping with the congruence principle, interviews were guided by the theoretical constructs contained within the MGDVPP, including parents’ attitudes, subjective norms, and perceived behavioral control related to promoting vegetable consumption in children (Hingle et al., 2012). The study discovered that the adapted model successfully identified parents’ motivations to encourage their children to eat more vegetables.

The use of the theory should be consistent with the basic goal of qualitative research, which is discovery. Alternatively stated, theories should be used as broad orienting frameworks for exploring topical areas without imposing preconceived ideas and biases. The theory should be consistent with the study findings and not be used to force-fit the researcher’s interpretation of the data (Sandelowski, 1993b). Divergence from the theory when it does not fit the study findings is illustrated in a qualitative study of hypertension prevention beliefs in Hispanics (Aroian, Peters, Rudner, & Waser, 2012). This study used the Theory of Planned Behavior as a guiding theoretical framework but found that coding separately for normative and control beliefs was not the best organizing schema for presenting the study findings. When divergence from the original theory occurs, the research report should explain and justify how and why the theory was modified (Bradbury-Jones et al., 2014).

Qualitative sampling methods should be “purposeful” (Coyne, 1997; Patton, 2015; Tuckett, 2004). Purposeful sampling is based on the study purpose and investigator judgments about which people and settings will provide the richest information for the research questions. The logic underlying this type of sampling differs from the logic underlying quantitative sampling (Patton, 2015). Quantitative research strives for empirical generalization. In qualitative studies, generalizability beyond the study sample is typically not the intent; rather, the focus is on deriving depth and context-embedded meaning for the relevant study population.

Purposeful sampling is a broad term. Theoretical sampling is one particular type of purposeful sampling unique to grounded theory methods (Coyne, 1997). In theoretical sampling, study participants are chosen according to theoretical categories that emerge from ongoing data collection and analyses (Bryant & Charmaz, 2010). Data collection and analysis are conducted concurrently to allow generating and testing hypotheses that emerge from analyzing incoming data. The following example from the previously mentioned qualitative interview study about transition from pediatric to adult care in adolescents with type 1 diabetes (Pierce et al., 2016) illustrates the process of theoretical sampling: An adolescent study participant stated that he was “turned off” by the “childish” posters in his pediatrician’s office. He elaborated that he welcomed transitioning to adult care because his diabetes was discovered when he was 18, an age when he reportedly felt more “mature” than most pediatric patients. These data were coded as “developmental misfit” and prompted a tentative hypothesis about developmental stage at entry for pediatric diabetes care and readiness for health care transition. Examining this hypothesis prompted seeking study participants who varied according to age or developmental stage at time of diagnosis to examine the theoretical relevance of an emerging theme about developmental fit.

Not all purposeful sampling, however, is “theoretical.” For example, ethnographic studies typically seek to understand a group’s cultural beliefs and practices (Creswell, 2013a). Consistent with this purpose, researchers conducting an ethnographic study might purposefully select study participants according to specific characteristics that reflect the social roles and positions in a given group or society (e.g., socioeconomic status, education; Johnson, 1990).

Random sampling is generally not used in qualitative research. Random selection requires a sufficiently large sample to maximize the potential for chance and, as will be discussed below, sample size is intentionally small in qualitative studies. However, random sampling may be used to verify or clarify findings (Patton, 2015). Validating study findings with a randomly selected subsample can be used to address the possibility that a researcher is inadvertently giving greater attention to cases that reinforce his or her preconceived ideas.

Regardless of the sampling method used, qualitative researchers should clearly describe the sampling strategy and justify how it fits the study when reporting study findings (transparency). A common error is to refer to theoretical sampling when the cases were not chosen according to emerging theoretical concepts. Another common error is to apply sampling principles from quantitative research (e.g., cluster sampling) to convince skeptical reviewers about the rigor or validity of qualitative research. Rigor is best achieved by being purposeful, making sound decisions, and articulating the rationale for those decisions. As mentioned earlier in the discussion of transferability, qualitative researchers are encouraged to describe their methods of sample selection and descriptive characteristics about their sample so that readers and reviewers can judge how the current sample may differ from others. Understanding the characteristics of each qualitative study sample is essential for the iterative nature of qualitative research whereby qualitative findings inform the development of future qualitative, quantitative, or mixed-methods studies. Reviewers should evaluate sampling decisions based on how they fit the study purpose and how they influence the quality of the end product.

No definitive rules exist about sample size in qualitative research. However, sample sizes are typically smaller than those in quantitative studies (Patton, 2015). Small samples often generate a large volume of data and information-rich cases, ultimately leading to insight regarding the phenomenon under study (Patton, 2015; Ritchie & Lewis, 2003). Sample sizes of 20–30 cases are typical, but a qualitative sample can be even smaller under some circumstances (Mason, 2010).

Sample size adequacy is evaluated based on the quality of the study findings, specifically the full development of categories and inter-relationships or the adequacy of information about the phenomenon under study (Corbin & Strauss, 2008; Ritchie & Lewis, 2003). Small sample sizes are of concern if they do not result in these outcomes. Data saturation (i.e., the point at which no new information, categories, or themes emerge) is often used to judge informational adequacy (Morgan, 1998; Ritchie & Lewis, 2003). Although enough participants should be included to obtain saturation (Morgan, 1998), informational adequacy pertains to more than sample size. It is also a function of the quality of the data, which is influenced by study participant characteristics (e.g., cognitive ability, knowledge, representativeness) and the researcher’s data-gathering skills and analytical ability to generate meaningful findings (Morse, 2015b; Patton, 2015).

Sample size is also influenced by type of qualitative research, the study purpose, the sample, the depth and complexity of the topic investigated, and the method of data collection. In general, the more heterogeneous the sample, the larger the sample size, particularly if the goal is to investigate similarities and differences by specific characteristics (Ritchie & Lewis, 2003). For instance, in a study to conduct an initial exploration of factors underlying parents’ motivations to use good parenting practices, theoretical saturation (i.e., the point at which no new information, categories, or themes emerge) was obtained with a small sample (n = 15), most likely because the study was limited to parents of young children (Hingle et al., 2012). If the goal of the study had been, for example, to identify racial/ethnic, gender, or age differences in food parenting practices, a larger sample would likely be needed to obtain saturation or informational adequacy.

Studies that seek to understand maximum variation in a phenomenon might also need a larger sample than one that is seeking to understand extreme or atypical cases. For example, a qualitative study of diet and physical activity in young Australian men conducted focus groups to identify perceived motivators and barriers to healthy eating and physical activity and examine the influence of body weight on their perceptions. Examining the influence of body weight status required 10 focus groups to allow for group assignment based on body mass index (Ashton et al., 2015). More specifically, 61 men were assigned to a healthy-weight focus group (n = 3), an overweight/obese focus group (n = 3), or a mixed-weight focus group (n = 4). Had the researcher not been interested in whether facilitators and barriers differed by weight status, its likely theoretical saturation could have been obtained with fewer groups. Depth of inquiry also influences sample size (Sandelowski, 1995). For instance, an in-depth analysis of an intervention for children with cancer and their families included 16 family members from three families. Study data comprised 52 hrs of videotaped intervention sessions and 10 interviews (West, Bell, Woodgate, & Moules, 2015). Depth was obtained through multiple data points and types of data, which justified sampling only a few families.

Authors of publications describing qualitative findings should show evidence that the data were “saturated” by a sample with sufficient variation to permit detailing shared and divergent perspectives, meanings, or experiences about the topic of inquiry. Decisions related to the sample (e.g., targeted recruitment) should be detailed in publications so that peer reviewers have the context for evaluating the sample and determining how the sample influenced the study findings (Patton, 2015).

When conducting qualitative research, voluminous amounts of data are gathered and must be prepared (i.e., transcribed) and managed. During the analytic process, data are systematically transformed through identifying, defining, interpreting, and describing findings that are meant to comprehensively describe the phenomenon or the abstract qualities that they have in common. The process should be systematic (dependability) and well-documented in the analysis section of a qualitative manuscript. For example, Kelly and Ganong (2011), in their study of medical treatment decisions made by families of children with cancer, described their analytic procedure by outlining their approach to coding and use of memoing (e.g., keeping careful notes about emerging ideas about the data throughout the analytic process), comparative analysis (e.g., comparing data against one another and looking for similarities and differences), and diagram drawing (e.g., pictorially representing the data structure, including relationships between codes).

Because the intent of qualitative research is to account for multiple perspectives, the goal of qualitative analysis is to comprehensively incorporate those perspectives into discernible findings. Researchers accustomed to doing quantitative studies may expect authors to quantify interrater reliability (e.g., kappa statistic) but this is not typical in qualitative research. Rather, the emphasis in qualitative research is on (1) training those gathering data to be rigorous and produce high-quality data and on (2) using systematic processes to document key decisions (e.g., code book), clear direction, and open communication among team members during data analysis. The goal is to make the most of the collective insight of the investigative team to triangulate or complement each other’s efforts to process and interpret the data. Instead of evaluating if two independent raters came to the same numeric rating, reviewers of qualitative manuscripts should judge to what extent the overall process of coding, data management, and data interpretation were systematic and rigorous. Authors of qualitative reports should articulate their coding procedures for others to evaluate. Together, these strategies promote trustworthiness of the study findings.

An example of how these processes are described in the report of a qualitative study is as follows:

The first two authors independently applied the categories to a sample of two interviews and compared their application of the categories to identify lack of clarity and overlap in categories. The investigators created a code book that contained a definition of categories, guidelines for their application, and excerpts of data exemplifying the categories. The first two authors independently coded the data and compared how they applied the categories to the data and resolved any differences during biweekly meetings. ATLAS.ti, version 6.2, was used to document and accommodate ongoing changes and additions to the coding structure (Palma et al., 2015, p. 224).

Multiple computer software packages for qualitative data analysis are currently available (Silver & Lewins, 2014; Yin, 2015). These packages allow the researcher to import qualitative data (e.g., interview transcripts) into the software program and organize data segments (e.g., delineate which interview excerpts are relevant to particular themes). Qualitative analysis software can be useful for organizing and sorting through data, including during the analysis phase. Some software programs also offer sophisticated coding and visualization capabilities that facilitate and enhance interpretation and understanding. For example, if data segments are coded by specific characteristics (e.g., gender, race/ethnicity), the data can be sorted and analyzed by these characteristics, which may contribute to an understanding of whether and/or how a particular phenomenon may vary by these characteristics.

The strength of computer software packages for qualitative data analysis is their potential to contribute to methodological rigor by organizing the data for systematic analyses (John & Johnson, 2000; MacMillan & Koenig, 2004). However, the programs do not replace the researchers’ analyses. The researcher or research team is ultimately responsible for analyzing the data, identifying the themes and patterns, and placing the findings within the context of the literature. In other words, qualitative data analysis software programs contribute to, but do not ensure scientific rigor or “objectivity” in, the analytic process. In fact, using a software program for analysis is not essential if the researcher demonstrates the use of alternative tools and procedures for rigor.

Qualitative papers sometimes combine results and discussion into one section to provide a cohesive presentation of the findings along with meaningful linkages to the existing literature (Burnard, 2004; Burnard, Gill, Stewart, Treasure, & Chadwick, 2008). Although doing so is an acceptable method for reporting qualitative findings, some journals prefer the two sections to be distinct.

When the journal style is to distinguish the two sections, the results section should describe the findings, that is, the themes, while the discussion section should pull the themes together to make larger-level conclusions and place the findings within the context of the existing literature. For instance, the findings section of a study of how rural African-American adolescents, parents, and community leaders perceived obesity and topics for a proposed obesity prevention program, contained a description of themes about adolescent eating patterns, body shape, and feedback on the proposed weight gain prevention program according to each subset of participants (i.e., adolescents, parents, community leaders). The discussion section then put these themes within the context of findings from prior qualitative and intervention studies in related populations (Cassidy et al., 2013). In the Discussion, when making linkages to the existing literature, it is important to avoid the temptation to extrapolate beyond the findings or to over-interpret them (Burnard, 2004). Linkages between the findings and the existing literature should be supported by ample evidence to avoid spurious or misleading connections (Burnard, 2004).

The results section of a qualitative research report is likely to contain more material than customary in quantitative research reports. Findings in a qualitative research paper typically include researcher interpretations of the data as well as data exemplars and the logic that led to researcher interpretations (Sandelowski & Barroso, 2002). Interpretation pertains to the researcher breaking down and recombining the data and creating new meanings (e.g., abstract categories, themes, conceptual models). Select quotes from interviews or other types of data (e.g., participant observation, focus groups) are presented to illustrate or support researcher interpretations. Researchers trained in the quantitative tradition, where interpretation is restricted to the discussion section, may find this surprising; however, in qualitative methods, researcher interpretations represent an important component of the study results. The presentation of the findings, including researcher interpretations (e.g., themes) and data (e.g., quotes) supporting those interpretations, adds to the trustworthiness of the study (Elo et al., 2014).

The Results section should contain a balance between data illustrations (i.e., quotes) and researcher interpretations (Lofland & Lofland, 2006; Sandelowski, 1998). Because interpretation arises out of the data, description and interpretation should be combined. Description should be sufficient to support researcher interpretations, and quotes should be used judiciously (Morrow, 2005; Sandelowski, 1994). Not every theme needs to be supported by multiple quotes. Rather, quotes should be carefully selected to provide “voice” to the participants and to help the reader understand the phenomenon from the participant’s perspective within the context of the researcher’s interpretation (Morrow, 2005; Ritchie & Lewis, 2003). For example, researchers who developed a grounded theory of sexual risk behavior of urban American Indian adolescent girls identified desire for better opportunities as a key deterrent to neighborhood norms for early sexual activity. They illustrated this theme with the following quote: “I don’t want to live in the ‘hood and all that…My sisters are stuck there because they had babies. That isn’t going to happen to me” (Saftner, Martyn, Momper, Loveland-Cherry, & Low, 2015, p. 372).

There is no precise formula for the proportion of description to interpretation. Both descriptive and analytic excess should be avoided (Lofland & Lofland, 2006). The former pertains to presentation of unedited field notes or interview transcripts rather than selecting and connecting data to analytic concepts that explain or summarize the data. The latter pertains to focusing on the mechanics of analysis and interpretation without substantiating researcher interpretations with quotes. Reviewer requests for methodological rigor can result in researchers writing qualitative research papers that suffer from analytic excess (Sandelowski & Barroso, 2002). Page limitations of most journals provide a safeguard against descriptive excess, but page limitations should not circumvent researchers from providing the basis for their interpretations.

Additional potential problems with qualitative results sections include under-elaboration, where themes are too few and not clearly defined. The opposite problem, over-elaboration, pertains to too many analytic distinctions that could be collapsed under a higher level of abstraction. Quotes can also be under- or over-interpreted. Care should be taken to ensure the quote(s) selected clearly support the theme to which they are attached. And finally, findings from a qualitative study should be interesting and make clear contributions to the literature (Lofland & Lofland, 2006; Morse, 2015b).

There is controversy over whether to quantify qualitative findings, such as providing counts for the frequency with which particular themes are endorsed by study participants (Morgan, 1993; Sandelowski, 2001). Qualitative papers usually report themes and patterns that emerge from the data without quantification (Dey, 1993). However, it is possible to quantify qualitative findings, such as in qualitative content analysis. Qualitative content analysis is a method through which a researcher identifies the frequency with which a phenomenon, such as specific words, phrases, or concepts, is mentioned (Elo et al., 2014; Morgan, 1993). Although this method may appeal to quantitative reviewers, it is important to note that this method only fits specific study purposes, such as studies that investigate the language used by a particular group when communicating about a specific topic. In addition, results may be quantified to provide information on whether themes appeared to be common or atypical. Authors should avoid using imprecise language, such as “some participants” or “many participants.” A good example of quantification of results to illustrate more or less typical themes comes from a manuscript describing a qualitative study of school nurses’ perceived barriers to addressing obesity with students and their families. The authors described that all but one nurse reported not having the resources they needed to discuss weight with students and families whereas one-quarter of nurses reported not feeling competent to discuss weight issues (Steele et al., 2011). If quantification of findings is used, authors should provide justification that explains how quantification is consistent with the aims or goals of the study (Sandelowski, 2001).

This article highlighted key theoretical and logistical considerations that arise in designing, conducting, and reporting qualitative research studies (see Table 1 for a summary). This type of research is vital for obtaining patient, family, community, and other stakeholder perspectives about their needs and interests, and will become increasingly critical as our models of health care delivery evolve. For example, qualitative research could contribute to the study of health care providers and systems with the goal of optimizing our health care delivery models. Given the increasing diversity of the populations we serve, qualitative research will also be critical in providing guidance in how to tailor health interventions to key characteristics and increase the likelihood of acceptable, effective treatment approaches. For example, applying qualitative research methods could enhance our understanding of refugee experiences in our health care system, clarify treatment preferences for emerging adults in the midst of health care transitions, examine satisfaction with health care delivery, and evaluate the applicability of our theoretical models of health behavior changes across racial and ethnic groups. Incorporating patient perspectives into treatment is essential to meeting this nation’s priority on patient-centered health care (Institute of Medicine Committee on Quality of Health Care in America, 2001). Authors of qualitative studies who address the methodological choices addressed in this review will make important contributions to the field of pediatric psychology. Qualitative findings will lead to a more informed field that addresses the needs of a wide range of patient populations and produces effective and acceptable population-specific interventions to promote health.

The authors thank Bridget Grahmann for her assistance with manuscript preparation.

This work was supported by National Cancer Institute of the National Institutes of Health (K07CA196985 to Y.W.). This work is a publication of the United States Department of Agriculture/Agricultural Research Center (USDA/ARS), Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, Texas. It is also a publication of the USDA/ARS, Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, Texas, and funded in part with federal funds from the USDA/ARS under Cooperative Agreement No. 58‐6250‐0‐008 (to D.T.). The contents of this publication do not necessarily reflect the views or policies of the USDA, nor does mention of trade names, commercial products, or organizations imply endorsement from the U.S. government. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of interest: None declared.